WIDELY USED DATA SET.¶
REFERENCES USED FROM DIFFERENT SOURCES IN OPEN DOMAIN:¶
PRIMARY REFERENCE:- https://medium.com/@rohanaggarwal45/thera-bank-case-with-univariate-as-well-as-bivariate-analysis-all-the-machine-learning-models-7f61d04eaa2a¶
STEP:-1 Read the column description and ensure you understand each attribute well¶
STEP:-2 Study the data distribution in each attribute, share your findings (15 Marks)¶
In [169]:
# Import Libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
In [170]:
# LOAD AND REVIEW DATA
pdata= pd.read_csv('Bank_Personal_Loan_Modelling.csv')
In [171]:
# GET THE SHAPE OF THE DATA TO UNDERSTAND THE SIZE
pdata.shape
Out[171]:
In [172]:
# CHECK THE FIRST FEW ROWS AND COLUMNS OF THE DATA-SET
pdata.head(10)
# FROM THE TABLE BELOW, IT CAN CLEARLY BE SEEN THAT THE FOLLOWING DISTRIBUTION OF VARIABLES OCCUR:
# A VERY USEFUL METHOD TO GENERATE REPORT SUMMARY IS AS BELOW
!pip install pandas_profiling
import pandas_profiling
pdata.profile_report()
# A COMPREHENSIVE DETAILED REPORT ABOUT THE DATA IS OUTPUT
Out[172]:
In [173]:
#CHECK FOR ANY NULL VALUES IN THE DATA
pdata.isnull().values.any()
#SINCE THE RESULT IS FALSE IT LOOKS LIKE THERE ARE NO NULL VALUES
# ALSO FROM THE WARNINGS WE CAN SEE THAT ZEROS ARE PRESENT ONLY IN EXPERIENCE, CCavg And Mortgage WHICH IS ALSO ACCEPTABLE AS IT
# IS A POSSIBLE SCENARIO
Out[173]:
In [174]:
pdata.describe()
Out[174]:
In [175]:
#JUST FOR HISTOGRAM PLOTS THE DATA IS STORED IN ANOTHER VARIABLE WITH THE OUTCOME VARIABLE and OTHER CATEGORICAL VARIABLES DROPPED
#pdata1=pdata.drop(['Personal Loan','Education','Experience','Family','Securities Account','Online','ZIP Code','CD Account','CreditCard'], axis=1)
pdata.hist(stacked=False,bins=100,figsize=(12,30),layout=(14,2));
In [176]:
#JUST FOR HISTOGRAM PLOTS THE DATA IS STORED IN ANOTHER VARIABLE WITH THE OUTCOME VARIABLE and OTHER CATEGORICAL VARIABLES DROPPED
pdata1=pdata.drop(['Personal Loan','Education','Experience','Family','Securities Account','Online','ZIP Code','CD Account','CreditCard'], axis=1)
pdata1.hist(stacked=False,bins=100,figsize=(12,30),layout=(14,2));
In [177]:
# THE NEXT STEP IS TO IDENTIFY CORRELATION IN DATA, THIS IS AGAIN DONE BY DROPPING ONLY THE OUTPUT COLUMN IN THE DATASET
pdata2=pdata.drop(['Personal Loan'], axis=1)
pdata2.corr()
# HERE IN THE TABLE WE CAN ALREADY OBSERVE NUMERICALLY THAT EXPERIENCE HAS A HIGH CORRELATION WITH AGE WITH A CORRELATION VALUE
# OF 0.99
Out[177]:
In [178]:
def plot_corr(df, size=20):
corr=df.corr()
fig, ax= plt.subplots(figsize=(size,size))
ax.matshow(corr)
plt.xticks(range(len(corr.columns)),corr.columns)
plt.yticks(range(len(corr.columns)),corr.columns)
In [179]:
# THIS STEP IS DONE TO VISUALIZE THE CORRELAITONS IN AN GRAPHICAL MANNER
plot_corr(pdata2)
# AGAIN WE OBSERVE THAT THE EXPERIENCE IS HIGHLY CORRELATED WITH AGE, SO IN THIS CASE WE CAN DROP EXPERIENCE AND i
In [180]:
# PLOTTING THE COMPLETE DATASET INCLUDING CATEGORICAL VARIABLES, SHOWS TOO MUCH INFORMATION WITHOUT ANY USE
sns.pairplot(pdata,diag_kind='kde')
Out[180]:
In [181]:
# PLOTTING THE COMPLETE DATASET AFTER DROPPING THE CATEGORICAL VARIABLES- LOT MORE EASIER TO UNDERSTAND
sns.pairplot(pdata1,diag_kind='kde')
# ALSO SHOWS THAT MORTGAGE HAS A SPIKE IN THE DATA SIMILAR TO AN OUTLIER
Out[181]:
EDUCATION VS PERSONAL LOAN ( EXAMINATION OF CATEGORICAL VARIABLES ON PERSONAL LOAN)
In [182]:
# NOW I AM GOING TO USE THE CROSS-TAB FUNCTION IN ORDER TO EXAMINE THE DISTRIBUTION
# OF EACH CATEGORICAL VARIABLE ON THE TARGET VARIABLE , for example what percent of graduates have
#availed personal loans etc
edu=pd.crosstab(pdata['Education'],pdata['Personal Loan'])
edu.div(edu.sum(1).astype(float),axis=0).plot(kind='bar',stacked=True)
print('cross tabulation can be given as:','\n',edu)
FAMILY VS PERSONAL LOAN
In [183]:
edu=pd.crosstab(pdata['Family'],pdata['Personal Loan'])
edu.div(edu.sum(1).astype(float),axis=0).plot(kind='bar',stacked=True)
print('cross tabulation can be given as:','\n',edu)
# AS CAN BE SEEN THE EFFECT OF FAMILY SIZE IS NOT AS MUCH AN IMPORTANT FACTOR
#DECIDING IF THE PERSONAL LOAN IS SANCTIONED
EXAMINATION OF BOOLEAN VARIABLES ON THE TARGET VARIABLE
CD_ACCOUNT vs PERSONAL LOAN
In [184]:
pdata.head()
Out[184]:
In [185]:
edu=pd.crosstab(pdata['CD Account'],pdata['Personal Loan'])
edu.div(edu.sum(1).astype(float),axis=0).plot(kind='bar',stacked=True)
print('cross tabulation can be given as:','\n',edu)
# AS CAN BE SEEN THE EFFECT OF CD ACCOUNT IS SHOWN BELOW, THOSE HAVING CD ACCOUNT
# HAVE MORE CHANCES OF AVAILING THE PERSONAL LOAN
ONLINE vs PERSONAL LOAN
In [186]:
edu=pd.crosstab(pdata['Online'],pdata['Personal Loan'])
edu.div(edu.sum(1).astype(float),axis=0).plot(kind='bar',stacked=True)
print('cross tabulation can be given as:','\n',edu)
# AS CAN BE SEEN THE EFFECT OF CREDIT CARD IS SHOWN BELOW, THOSE HAVING
# ONLINE BANKING IS NOT HAVING ANY EFFECT ON PERSONAL LOAN
Securities_Account vs Personal Loan
In [187]:
edu=pd.crosstab(pdata['Securities Account'],pdata['Personal Loan'])
edu.div(edu.sum(1).astype(float),axis=0).plot(kind='bar',stacked=True)
print('cross tabulation can be given as:','\n',edu)
# AS CAN BE SEEN THE EFFECT OF CREDIT CARD IS SHOWN BELOW, THOSE HAVING
# Securities Account IS NOT HAVING ANY EFFECT ON PERSONAL LOAN
Numerical Variables vs Personal Data
In [188]:
# WE WILL TRY TO LOOK AT THE MEAN OF THE NUMERIC INDEPENDENT VARIABLE AND THEN
# COMPARE IT WITH THE TARGET VARIABLE BY USING THE GROUPBY FUNCTION
ax1=pdata.groupby('Personal Loan')['Age'].mean().plot(kind='bar')
ax1.set_ylabel('Age')
# WHAT IS HAPPENING IN THE BELOW PLOT IS THAT THE Y-AXIS CONSISTS OF THE MEAN AGE
# OF THE CUSOTMERS AND IT IS PLOTTED AGAINST THE PEOPLE WHO HAVE AND HAVE NOT AVAILED
# THE PERSONAL LOAN AND AS CAN SEEN , WE DO NOT SEE ANY MAJOR CHANGE IN MEAN AGE.
# WE MAY HAVE TO BREAK THE AGES IN TO BINS AND THEN REVIEW THE DATA AGAIN.
Out[188]:
In [189]:
# BREAK THE AGE DATA SET IN TO SEVERAL BINS/CATEGORIES
#bins = [25,35,45,55,65] # from pdata.describe() we can get the range of the age of the people and then
#suitably arrange the bins
#pdata_age=pdata
#group = ['Very Young','Youth','middleage','oldage']
#pdata_age['Age_bin']=pd.cut(pdata_age['Age'],bins,labels=group)
#age=pd.crosstab(pdata_age['Age_bin'],pdata_age['Personal Loan'])
#age.div(age.sum(1).astype(float),axis=0).plot(kind='bar',stacked=True)
# As can be seen below the age distribution does not affect the personal loan distribution
# PLEASE NOTE THAT AS AGE AND EXPERIENCE ARE CLOSELY CORRELATED WE CAN EXPECT
# THE SAME RESULTS.
pdata.head()
Out[189]:
In [ ]:
RELATION BETWEEN CREDIT CARD USAGE AND PERSONAL LOAN
In [190]:
ax1=pdata.groupby('Personal Loan')['CCAvg'].mean().plot(kind='bar')
ax1.set_ylabel('Credit Card Spending')
# AS CAN BE SEEN FROM THE BAR PLOT BELOW HIGHER THE CREDIT CARD SPENDING
# MORE LIKELY IS THE PERSONAL LOAN AVAILING
Out[190]:
INCOME VS PERSONAL LOAN
In [191]:
ax=pdata.groupby('Personal Loan')['Income']
ax1=ax.mean().plot(y='Income',kind='bar')
ax1.set_ylabel('Income')
# AS CAN BE SEEN FROM THE PLOT BELOW MORE PEOPLE WITH HIGHER INCOME AVAIL THE
# PERSONAL LOAN WHICH IS TO BE EXPECTED
Out[191]:
In [192]:
# OUTLIER EVALUATION IN THE MORTGAGE VARIABLE
# AS WE HAD DESCRIBED IN THE PAIR PLOT AND THE DESCRIBE FUNCTION THE MORTGAGE
# VARIABLE HAS OUTLIERS IN THE DATA DISTRIBUTION
# THE APPROACH TAKEN HERE IS THAT THE ROWS WITH MORTGAGE VARIABLE WITH Z SCORE
# LESS THAN 3 IS TAKEN.
from scipy import stats
pdata['Mortgage_zscore']=np.abs(stats.zscore(pdata['Mortgage']))
pdata3=pdata[pdata['Mortgage_zscore']<3]
pdata3.drop('Mortgage_zscore',axis=1,inplace=True)
pdata3.shape
sns.pairplot(pdata3,diag_kind='kde')
# NOW NOTE THAT THE HIGHEST VALUE OF THE OUTLIER EEARLIER WAS AROUND 600 AND IT IS NOW 300 SO THE EFFECT OF THE OUTLIER IS BEING MINIMIZED
Out[192]:
STEP:3 Get the target column distribution. Your comments (5 marks)¶
In [193]:
# NOW WE WILL TRY TO SEE HOW MANY PEOPLE HAVE AVAILED PERSONAL LOAN IN THE
# PRESENT DATA SET
n_personal_loan_takers=len(pdata.loc[pdata['Personal Loan']==1])
n_personal_loan_non_takers=len(pdata.loc[pdata['Personal Loan']==0])
print("The Number of people who have availed Personal Loans:{0} ({1:2.2f}%)".format(n_personal_loan_takers,(n_personal_loan_takers/(n_personal_loan_takers + n_personal_loan_non_takers))*100 ))
print("The Number of people who have not availed Personal Loans:{0} ({1:2.2f}%)".format(n_personal_loan_non_takers,(n_personal_loan_non_takers/(n_personal_loan_takers + n_personal_loan_non_takers))*100 ))
AS CAN BE SEEN ABOUT 10% OF THE PEOPLE HAVE TAKEN PERSONAL LOANS AND ABOUT 90% HAVE NOT AVAILED IT
STEP:4 Split the data into training and test set in the ratio of 70:30 respectively (5 marks)¶
In [194]:
# BEFORE PROCEEDING IT IS IMPORTANT TO DROP THE FOLLOWING VARIABLES FROM THE DATA SET
# 1.) EXPERIENCE- THIS VARIABLE IS SHOWING HIGH CORRELATION WITH AGE AND HENCE CAN BE DROPPED
# 2.) ID - THIS IS DATA WITH UNIQUE VALUE AND JUST REPRESENTS SERIAL NUMBER
# 3.) zIP CODE - ALSO A CATEGORICAL VARIABLE REPRESENTING A SERIAL NUMBER
# NOTE THAT WE ARE NOW USING THE DATA SET PDATA3 WHEREIN OUTLIERS ARE TAKEN CARE OF
pdata4=pdata3.drop(['Experience','ZIP Code','ID'], axis=1)
pdata4.head()
Out[194]:
In [195]:
from sklearn.model_selection import train_test_split
X=pdata4.drop('Personal Loan',axis=1) # Dropping the dependent variable and keeping all the
Y= pdata4['Personal Loan'] # PREDICTED CLASS (1= LOAN AVAILED AND 0= LOAN NOT AVAILED)
x_train,x_test,y_train,y_test=train_test_split(X,Y, test_size=0.3,random_state=1)
x_train.head(10)
y_train.head(10)
Out[195]:
In [196]:
# CHECKING THE SPLIT OF THE DATA
In [197]:
print("{0:0.2f}% data is in the training set".format((len(x_train)/len(pdata4.index))*100))
print("{0:0.2f}% data is in the testing set".format((len(x_test)/len(pdata4.index))*100))
In [198]:
# NOW WE HAVE TO CHECK THE SPLIT OF THE DATA ( THOSE WHO HAVE AVAILED AS COMPARED TO THOSE WHO HAVE NOT) IN THE TESTING AND
# TRAINING SET
In [199]:
# NOW WE WILL TRY TO SEE HOW MANY PEOPLE HAVE AVAILED PERSONAL LOAN IN THE
# PRESENT DATA SET
n_personal_loan_takers=len(pdata.loc[pdata['Personal Loan']==1])
n_personal_loan_non_takers=len(pdata.loc[pdata['Personal Loan']==0])
print("ORIGINAL DATASET- who have availed Personal Loans:{0} ({1:2.2f}%)".format(n_personal_loan_takers,(n_personal_loan_takers/(n_personal_loan_takers + n_personal_loan_non_takers))*100 ))
print("ORIGINAL DATASET who have not availed Personal Loans:{0} ({1:2.2f}%)".format(n_personal_loan_non_takers,(n_personal_loan_non_takers/(n_personal_loan_takers + n_personal_loan_non_takers))*100 ))
print("")
###########TRAINING DATASET#####################
n_personal_loan_takers_Training=len(y_train[y_train[:]==1])
n_personal_loan_non_takers_Training=len(y_train[y_train[:]==0])
print("TRAINING DATASET- who have availed Personal Loans:{0} ({1:2.2f}%)".format(n_personal_loan_takers_Training,(n_personal_loan_takers_Training/(n_personal_loan_takers_Training + n_personal_loan_non_takers_Training))*100 ))
print("TRAINING DATASET who have not availed Personal Loans:{0} ({1:2.2f}%)".format(n_personal_loan_non_takers_Training,(n_personal_loan_non_takers_Training/(n_personal_loan_takers_Training + n_personal_loan_non_takers_Training))*100 ))
print("")
###########TESTING DATASET#####################
n_personal_loan_takers_Testing=len(y_test[y_test[:]==1])
n_personal_loan_non_takers_Testing=len(y_test[y_test[:]==0])
print("TESTING DATASET- who have availed Personal Loans:{0} ({1:2.2f}%)".format(n_personal_loan_takers_Testing,(n_personal_loan_takers_Testing/(n_personal_loan_takers_Testing + n_personal_loan_non_takers_Testing))*100 ))
print("TESTING DATASET who have not availed Personal Loans:{0} ({1:2.2f}%)".format(n_personal_loan_non_takers_Testing,(n_personal_loan_non_takers_Testing/(n_personal_loan_takers_Testing + n_personal_loan_non_takers_Testing))*100 ))
print("")
In [200]:
# AS CAN BE SEEN CLEARLY, THE SPLIT B/W THE LOAN TAKERS AND NON LOAN TAKERS IN THE TRAINING AND TESTING DATASET ARE ALSO KEPT
# THE SAME.
LOGISTIC REGRESSION¶
In [201]:
from sklearn import metrics
from sklearn.linear_model import LogisticRegression
# Fit the model on train
model= LogisticRegression(solver="liblinear")
model.fit(x_train,y_train)
#Predict on Test
y_predict = model.predict(x_test)
coef_df=pd.DataFrame(model.coef_) # converts the coefficient matrix in to a dataframe
coef_df['intercept']=model.intercept_
print(coef_df)
In [202]:
#EVALUATE MODEL SCORE - TEST SCORE
model_score = model.score(x_test,y_test)
print(model_score)
In [203]:
# The model score on test data is close to 95% which is very good.
CONFUSION MATRIX- LOGISTIC REGRESSION¶
In [204]:
cm=metrics.confusion_matrix(y_test,y_predict,labels=[1,0])
df_cm=pd.DataFrame(cm,index=[i for i in ["1","0"]],
columns=[i for i in ["Predict 1","Predict 0"]])
plt.figure(figsize=(7,5))
sns.heatmap(df_cm,annot=True)
Out[204]:
In [205]:
#THE CONFUSION MATRIX FOR LOGISTIC REGRESSION IS SHOWN AS ABOVE:
#TRUE POSITIVES (TP): THE MODEL CORRECTLY PREDICTED THAT 66 PEOPLE WILL AVAIL THE PERSONAL LOAN
#TRUE NEGATIVES (TN): THE MODEL CORRECTLY PREDICTED THAT 1300 PEOPLE WILL NOT AVAIL THE PERSONAL LOAN
#FALSE POSITIVE (FP): THE MODEL PREDICTED THAT 15 PEOPLE WILL AVAIL LOAN BUT THEY DID NOT - TYPE I ERROR
#FALSE NEGATIVE (FN): THE MODEL PREDICTED THAT 62 PEOPLE WILL NOT AVAIL THE LOAN BUT THEY DID END UP AVAILING THE PERSONAL LOAN - TYPE -II ERROR ( MORE IMPORTANT IN THIS CASE)
In [206]:
#EVALUATION OF THE LOGISTIC REGRESSION MODEL
print("Classification Report")
print (metrics.classification_report(y_test,y_predict,labels=[1,0]))
In [207]:
# AS CAN BE SEEN ABOVE THE PRECISION IS ABOUT 81% AND RECALL IS ABOUT 52%
# WHICH MEANS THAT OUT OF 100 POSITIVE IDENTITIFICATION, WE CAN BE SURE THAT 81 TIMES WE GOT THE RIGHT PERSON WHO CAN AVAIL THE LOAN
# NOW WE ARE ABOUT 52% ACCURATE IN IDENTIFYING THE PEOPLE LIKELY IN AVAILING THE LOAN.
# SUPPORT FOR 1 IS 128 WHICH MEANS , 128 PEOPLE ARE LIKELY TO AVAIL THE PERSONAL LOAN
NAIVE BAYES ALGORITHM¶
In [208]:
# SPLIT DATA
from sklearn.model_selection import train_test_split
X=pdata4.drop('Personal Loan',axis=1) # Dropping the dependent variable and keeping all the
Y= pdata4['Personal Loan'] # PREDICTED CLASS (1= LOAN AVAILED AND 0= LOAN NOT AVAILED)
x_train1,x_test1,y_train1,y_test1=train_test_split(X,Y, test_size=0.3,random_state=1)
x_train1.head(10)
y_train1.head(10)
Out[208]:
In [209]:
from sklearn.naive_bayes import GaussianNB # using Gaussian algorithm from Naive Bayes
#create the model
Personal_Loan_Model=GaussianNB()
Personal_Loan_Model.fit(x_train1,y_train1.ravel())
Out[209]:
PERFORMANCE OF OUR MODEL WITH TRAINING DATA
In [210]:
Personal_Loan_Train_Predict = Personal_Loan_Model.predict(x_train1)
from sklearn import metrics
print ("Model Accuracy: {0:.4f}".format(metrics.accuracy_score(y_train1,Personal_Loan_Train_Predict)))
print()
PERFORMANCE OF OUR MODEL WITH TESTING DATA¶
In [211]:
Personal_Loan_Test_Predict = Personal_Loan_Model.predict(x_test1)
from sklearn import metrics
print ("Model Accuracy: {0:.4f}".format(metrics.accuracy_score(y_test1,Personal_Loan_Test_Predict)))
print()
CONFUSION MATRIX- NAIVE BAYES¶
In [212]:
print ("Confusion Matrix")
cm=metrics.confusion_matrix(y_test1, Personal_Loan_Test_Predict, labels=[1,0])
df_cm=pd.DataFrame(cm,index=[i for i in ["1","0"]],
columns=[i for i in ["Predict 1","Predict 0"]])
plt.figure(figsize=(7,5))
sns.heatmap(df_cm,annot=True)
Out[212]:
In [213]:
#THE CONFUSION MATRIX FOR NAIVE BAIYES IS SHOWN AS ABOVE:
#TRUE POSITIVES (TP): THE MODEL CORRECTLY PREDICTED THAT 75 PEOPLE WILL AVAIL THE PERSONAL LOAN
#TRUE NEGATIVES (TN): THE MODEL CORRECTLY PREDICTED THAT 1200 PEOPLE WILL NOT AVAIL THE PERSONAL LOAN
#FALSE POSITIVE (FP): THE MODEL PREDICTED THAT 100 PEOPLE WILL AVAIL LOAN BUT THEY DID NOT - TYPE I ERROR
#FALSE NEGATIVE (FN): THE MODEL PREDICTED THAT 53 PEOPLE WILL NOT AVAIL THE LOAN BUT THEY DID END UP AVAILING THE PERSONAL LOAN - TYPE -II ERROR ( MORE IMPORTANT IN THIS CASE)
In [214]:
#EVALUATION OF THE NAIVE BAYES MODEL
print("Classification Report")
print (metrics.classification_report(y_test1, Personal_Loan_Test_Predict,labels=[1,0]))
In [215]:
# AS CAN BE SEEN ABOVE THE PRECISION IS ABOUT 42% AND RECALL IS ABOUT 59%
# WHICH MEANS THAT OUT OF 100 POSITIVE IDENTITIFICATION, WE CAN BE SURE THAT 42 TIMES WE GOT THE RIGHT PERSON WHO CAN AVAIL THE LOAN
# NOW WE ARE ABOUT 59% ACCURATE IN IDENTIFYING THE PEOPLE LIKELY IN AVAILING THE LOAN.
# SUPPORT IS 128 WHICH MEANS , 128 PEOPLE ARE LIKELY TO AVAIL THE PERSONAL LOAN, HERE WE ARE SEEING THAT RECALL SCORE HAS INCREASED
# AT THE EXPENSE OF PRECISION, SO WE SEE THAT FALSE NEGATIVES HAVE GONE DOWN WHICH IS MORE IMPORTANT.
kNN MODEL¶
In [216]:
from sklearn.model_selection import train_test_split
X=pdata4.drop('Personal Loan',axis=1) # Dropping the dependent variable and keeping all the
Y= pdata4['Personal Loan'] # PREDICTED CLASS (1= LOAN AVAILED AND 0= LOAN NOT AVAILED)
x_train2,x_test2,y_train2,y_test2=train_test_split(X,Y, test_size=0.3,random_state=1)
x_train2.head(10)
y_train2.head(10)
Out[216]:
In [217]:
#BUILD kNN MODEL
from sklearn.neighbors import KNeighborsClassifier
NNH=KNeighborsClassifier(n_neighbors=5, weights ='distance')
In [218]:
# CALL NEAREST NEIGHBOR ALGORITHM
NNH.fit(x_train2,y_train2)
Out[218]:
In [219]:
#EVALUATE THE PERFORMANCE OF kNN MODEL-test data
In [220]:
Personal_Loan_Test_Predict2=NNH.predict(x_test2)
NNH.score(x_test2,y_test2)
Out[220]:
In [221]:
#EVALUATE THE PERFORMANCE OF kNN MODEL-train data , since it is 1 looks like some overfittig is happening
In [222]:
Loan_Predict2=NNH.predict(x_train2)
NNH.score(x_train2,y_train2)
Out[222]:
CONFUSION MATRIX - kNN ALGORITHM¶
In [223]:
print ("Confusion Matrix")
cm=metrics.confusion_matrix(y_test2, Personal_Loan_Test_Predict2, labels=[1,0])
df_cm=pd.DataFrame(cm,index=[i for i in ["1","0"]],
columns=[i for i in ["Predict 1","Predict 0"]])
plt.figure(figsize=(7,5))
sns.heatmap(df_cm,annot=True)
Out[223]:
In [224]:
#THE CONFUSION MATRIX FOR kNN IS SHOWN AS ABOVE:
#TRUE POSITIVES (TP): THE MODEL CORRECTLY PREDICTED THAT 50 PEOPLE WILL AVAIL THE PERSONAL LOAN
#TRUE NEGATIVES (TN): THE MODEL CORRECTLY PREDICTED THAT 1300 PEOPLE WILL NOT AVAIL THE PERSONAL LOAN
#FALSE POSITIVE (FP): THE MODEL PREDICTED THAT 45 PEOPLE WILL AVAIL LOAN BUT THEY DID NOT - TYPE I ERROR
#FALSE NEGATIVE (FN): THE MODEL PREDICTED THAT 78 PEOPLE WILL NOT AVAIL THE LOAN BUT THEY DID END UP AVAILING THE PERSONAL LOAN - TYPE -II ERROR ( MORE IMPORTANT IN THIS CASE)
In [225]:
#EVALUATION OF THE kNN MODEL
print("Classification Report")
print (metrics.classification_report(y_test2, Personal_Loan_Test_Predict2,labels=[1,0]))
In [226]:
# AS CAN BE SEEN ABOVE THE PRECISION IS ABOUT 53% AND RECALL IS ABOUT 39%
# WHICH MEANS THAT OUT OF 100 POSITIVE IDENTITIFICATION, WE CAN BE SURE THAT 53 TIMES WE GOT THE RIGHT PERSON WHO CAN AVAIL THE LOAN
# NOW WE ARE ABOUT 39% ACCURATE IN IDENTIFYING THE PEOPLE LIKELY IN AVAILING THE LOAN.
# SUPPORT IS 128 WHICH MEANS , 128 PEOPLE ARE LIKELY TO AVAIL THE PERSONAL LOAN, HERE WE ARE SEEING THAT RECALL SCORE HAS DECREASED
# AT THE EXPENSE OF PRECISION, SO WE SEE THAT FALSE NEGATIVES HAVE GONE DOWN WHICH IS MORE IMPORTANT.
STEP 7 : Give your reasoning on which is the best model in this case and why it performs better? (5 marks)¶
In [227]:
# let us us summarize our findings:
#MODEL SCORE OF THE MODELS
#1.) LOGISTIC REGRESSION - 95% ----HIGHEST
#2.) NAIVE BAYES - 89%
#3.) kNN - 92%
# BUT THE DECISION HERE CANNOT BE BASED ONLY ON THE ACCURACY ALONE AS WE NEED TO UNDERSTAND WHICH PARAMETER SHOULD BE CHOSEN FOR EVLAUAITON
# LET US LOOK AT PRECISION VALUE HERE
#1.) LOGISTIC REGRESSION - 81%--- HIGHEST
#2.) NAIVE BAYES - 42%
#3.) kNN - 53%
# PRECISION GIVES US AN IDEA OF WHAT PROPORTIONS OF POSITIVE IDENTIFICATIONS WERE CORRECT
# LET US LOOK AT RECALL VALUE HERE
#1.) LOGISTIC REGRESSION - 52%
#2.) NAIVE BAYES - 59% -- HIGHEST
#3.) kNN - 39%
# RECALL HERE IS AN IMPORTANT MEASURE OF WHAT PROPORTIONS OF POSITIVES WERE IDENTIFIED CORRECTLY ?
# AND IN THIS CASE IT IS IMPORTANT FOR US TO MEASURE THIS AS WE CANNOT AFFORD TO LET GO OF PEOPLE WHO
# WOULD HAVE AVAILED LOAN BUT WERE NOT IDENTIFIED POSITIVELY.
# ONE FINAL PARAMETER WE WILL LOOK AT IS THE NUMBER OF TRUE POSITIVES OR THE NUMBER OF POSITIVES :
#1.) LOGISTIC REGRESSION - 66 TRUE POSITIVE CASES WERE IDENTIFIED
#2.) NAIVE BAYES - 75 TRUE POSITIVE CASES WERE IDENTIFIED
#3.) kNN - 50 TRUE POSITIVE CASES WERE IDENTIFIED
# BASED ON THE RESULTS SUMMARY ABOVE WE CAN SEE THAT EVEN THOUGH THE MODEL ACCURACY SCORE IS MORE IN LOGISTIC REGRESSION,
# THE NUMBER OF TRUE POSITIVE CASES AND RECALL RATE IS HIGHEST IN NAIVE BAYES APPROACH....
# IN THIS CASES WE NEED TO PAY MORE ATTENTION TO RECALL RATE AS WE CANNOT AFFORD TO NEGATIVELY IDENTIFY CUSTOMERS
# WHO WOULD HAVE AVAILED THE PERSONAL LOAN